Fix all luminance formulas I could find#5
Open
Filoppi wants to merge 1 commit intomchaptel:nightlyfrom
Open
Conversation
The "XYZ_RGB" matrices remain very very very slightly different (like 0.212671 vs 0.2126) from the luma vector I used though it shouldn't really matter.
|
overall all the shaders should be checked whether those values are used on linear or gamma encoded values |
Owner
|
most of those would be better replaced by calls to the helper function with the correct values I think.. |
EndlesslyFlowering
approved these changes
Mar 19, 2023
EndlesslyFlowering
left a comment
EndlesslyFlowering
left a comment
There was a problem hiding this comment.
I guess it's not used anywhere, so it's fine too.
changing all shaders to work in linear space will be a huge task.
|
|
||
| float get_luma(float3 col) { | ||
| return max(dot(col, float3(0.299, 0.587, 0.114)), 0.001); | ||
| return max(dot(col, float3(0.2126, 0.7152, 0.0722)), 0.001); |
There was a problem hiding this comment.
this seems to be for use on gamma encoded values
those should be converted to linear first and this can't be used
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The "XYZ_RGB" matrices remain very very very slightly different (like 0.212671 vs 0.2126) from the luma vector I used though it shouldn't really matter.
As previously discussed, the game was using the BT.601 luminance formula all around, instead of the BT.709, which is the color space it actually uses. Likely a mistake that carried over from the 90s.
Mind that many game shaders still use the wrong luminance formula even with these fixes.